Spike-Based Vision Processing. Seeing without Frames

Conventional vision sensing and processing is based on capturing sequences of still frames and process them (or a subset of them) one by one. For sophisticated vision processing usually a complicated sequence of 2D operations is required: different convolution operations, edge detections and extraction, computation of orientations, grouping of segments, contour detections and segmentations, morphological operations, object recognition. Each of these operations are computationally expensive since they are applied on all pixels of the captured and processed images. The vision system of living animals does not operate on acquiring and processing sequences of still frames. Biological vision systems exploit spike-based processing. Also, biological brains are made with a slow ‘technology’ which uses neurons whose response times are in the order of milliseconds. Nonetheless, the computational power of biological brains overwhelmingly outperforms present day fastest computers for tasks like vision (or sensory processing in general). The reason for this could be, on one side the massive parallelism and high number of processing units, but on the other side could also be justified by a different type of visual information coding, transmission and computation, much more efficient than frame based processing. In the already concluded european project CAVIAR1 (IST2001-34124, www.imse.cnm.es/caviar) we have performed preliminary explorations of the powerful principles of vision sensing and processing using spiking events, through the use of the incipient AER (Address Event Representation) technology. Biological cortical structures are composed of a small number of processing layers (no more than 10), as shown in Fig. 1. Each neuron in a layer connects to a ‘projection field’ in the next layer. Each connection in the projection field has a given ‘strength’ or ‘weight’, and the set of weights of the projection field is fixed for a given layer (it does not change with position of sending neuron). Consequently, this is mathematically equivalent to convolution operations, where the kernel of the convolution is the set of weights of the projection field. In the topology of Fig. 1, electrical spikes are sent from neurons in a layer to the neurons in the next layer through their respective projections fields. Spikes are asynchronous, and each neuron in a layer decides when to send a spike, depending on the history of spikes it has received. Therefore, in such a